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DETAILED ACTION 

1 . The Office withdraws the previous rejections of the claims under 35 USC § 1 12-2 nd 
paragraph, in light of the amendment. However, the Office maintains the previous rejections of 
the claims under 35 USC § 103(a), in light of the amendment. 

Response to Arguments 

2. Applicant's arguments filed 4/22/2008 have been fully considered but they are not 
persuasive. 

The previous claim objections, and rejections under 35 USC §1 12-2 nd paragraph have 
been withdrawn. 

Regarding the previous rejection of the claims under 35 USC § 103(a), Applicant argues 
on page 8 that the Klemmer reference was improperly used in a 103(a) rejection because is not 
prior art under 35 USC § 102(b). 

The Office respectfully disagrees that the use of Klemmer was improper under 35 USC 
§ 103(a). It is unclear why Applicant argues that only § 102(b) references may be used in a 
§ 103(a) rejection. There is no such requirement. Additionally, no support was set forth for such 
a requirement. 
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Regarding the previous rejection of independent claims 1 and 25 under 35 USC 103(a), 
Applicant argues on page 9 that the Klemmer reference does not teach "a plurality of user 
selectable identifiers representing the features extracted from the media content for selection by a 
user to play media content segments of a defined length associated with each of the features, 
wherein the plurality of selectable of identifiers are linked to locations on the timeline'", because 
Klemmer discloses barcodes presented next to sections of a book. 

The Office respectfully disagrees. Klemmer teaches/suggests the limitations as recited. 
For example, Klemmer discusses the adding of timccode information to text in the last paragraph 
of page 91, and the association/linking of selectable barcodes and chronological 
sections/locations of a book. These selectable barcodes are thus linked to locations on a timeline 
of a book. 

Regarding the previous rejection of independent claims 1 and 25 under 35 USC 103(a), 
Applicant argues on page 10 that the cited references does not teach the recited claim limitations, 
including formatted media that includes a graphical representation of a timeline and wherein the 
plurality of selectable of identifiers are linked to locations on the timeline". 

The Office respectfully disagrees. Klemmer teaches/suggests the limitations as recited. 
For example, Klemmer discusses the adding of timecode information to text in the last paragraph 
of page 91, and the association/linking of selectable barcodes and chronological 
sections/locations of a book. These selectable barcodes are thus linked to locations on a timeline 
of a book. Additionally, the choice of data as picture/graphics or text was an obvious variant. 
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Regarding the previous rejection of the claims dependent upon claims 1 and 25 under 35 
USC 103(a), Applicant argues on page 1 1 that these claims are allowable for the arguments set 
forth above. 

The Office respectfully disagrees, and re-asserts the counter-arguments set forth above. 

It is further noted that any citation to specific, pages, columns, lines, or figures in the 
prior art references and any interpretation of the references should not be considered to be 
limiting in any way. A reference is relevant for all it contains and may be relied upon for all that 
it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 
1331, 1332-1333, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 
1006, 1009, 158 USPQ 275, 277 (CCPA 1968)). 

The Office also notes MPEP § 2144.01, that quotes In re Preda, 401 F.2d 825, 159 USPQ 
342, 344 (CCPA 1968) as stating "in considering the disclosure of a reference, it is proper to take 
into account not only specific teachings of the reference but also the inferences which one skilled 
in the art would reasonably be expected to draw therefrom." Further MPEP 2123, states that "a 
reference may be relied upon for all that it would have reasonably suggested to one having 
ordinary skill the art, including nonpreferred embodiments. Merck & Co. v. Biocraft 
Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert, denied, 493 U.S. 975 (1989). 
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For at least these reasons, the Office asserts the rejections of the claims as set forth 

below. 



Claim Rejections - 35 USC §103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-6, 8, 10, 12-22, 25, 27, 31, 33-42 and 44-45 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Scott R. Klemmer et al. ("Books With Voices: Paper 
Transcripts as a Tangible Interface to Oral Histories", CHI 2003 , Fort Lauderdale, FL, Apr. 5-10, 
2003, pp. 89-96, hereafter referred to as "Klemmer") in view of Graham et al. (US Patent No. 
6,369,81 1, filed Sep. 9, 1998 and issued Apr. 9, 2002, hereafter referred to as "Graham"). 

Regarding independent claim 1 : Klemmer teaches A computer system for generating 
a representation of time-based media, the system comprising: a feature extraction module for: 
extracting features from media content; (See Klemmer page 92 in the top paragraph of the right 
column discussing the creating of an MPEG-2 video from a video source.) and generating a 
media representation representing the features extracted; (See Klemmer page 92 in the top 
paragraph of the right column discussing the making of corresponding JPEG thumbnails.) a 
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formatting module for formatting the media representation generated, the formatting module 
being communicatively coupled to the feature extraction module to apply features extracted to 
the media representation, wherein the formatting module formats the media representation 
according to a representation specification; (See Klemmer page 92 in the top paragraph of the 
right column discussing the creating of a paper layout from a time stamped transcript.) wherein 
the formatted media representation includes a graphical representation of a timeline and a 
plurality of user selectable identifiers representing the features extracted from the media 
content for selection by a user to play media content segments of a defined length associated 
with each of the features, wherein the plurality of selectable identifiers are linked to locations 
on the timeline. (See Klemmer page 91 in the last paragraph of the right column discussing the 
adding of time code information to the print format. See also page 92 Fig. 3 showing barcodes 
linking chronological sections of a book to A/V data and page 92 in the top paragraph of the 
right column discussing the creating of a paper layout from a time stamped transcript.) 

However, Klemmer does not explicitly teach the remaining limitations as claimed. 
Graham, though, discloses and a printer for printing the formatted media representation, the 
printer being communicatively coupled to the formatting module to receive instructions for 
printing a document displaying the formatted media representation, (See Graham Fig. 4A 
#408 and col. 5 lines 32-50 teaching the printing of a paper reader's assistant document having 
an imprinted thumbnail and teaching additional information as being a "discussion" of user 
interests.) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Graham for the benefit of Klemmer, because to do so allowed a 
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designer to implement a mechanism to personalize a document for a particular user, as taught by 
Graham in the Abstract. These references were all applicable to the same field of endeavor, i.e., 
annotation of paper documents with electronic information. 

Regarding claim 2: Klemmer teaches module further comprises content recognition 
software for recognizing features in the media content. (See Klemmer page 92 in the top 
paragraph of the right column discussing the creating of an MPEG-2 video from a video source.) 

Regarding claim 3: Klemmer does not explicitly teach the remaining limitations as 
claimed. Graham, though, discloses processing logic for controlling a printer driver interface 
associated with the printer. (See Graham col. 6 lines 4-7 discussing the use of Postscript 
printing.) 

Regarding claim 4: Klemmer does not explicitly teach the remaining limitations as 
claimed. Graham, though, discloses processing logic for controlling a printer console on the 
printer. (See Graham col. 6 lines 4-7 discussing the use of Postscript printing.) 

Regarding claim 5: Klemmer teaches wherein the feature extraction module is further 
adapted to generate the media representation in digital format. (See Klemmer page 92 in the 
top paragraph of the right column discussing the creating of an MPEG-2 video from a video 
source.) 
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Regarding claim 6: Klemmer teaches wherein the feature extraction module is further 
adapted to generate the media representation in paper format. (See Klemmer page 92 Fig. 3 
and the top paragraph in the right column teaching a video paper system.) 

Regarding claim 8: Klemmer teaches wherein at least one of the user-selectable 
identifiers comprises a barcode printed on the document displaying the media representation. 

(See Klemmer page 92 Fig. 3 showing the incorporation of barcodes on a video paper system 
document.) 

Regarding claim 10: Klemmer teaches wherein the barcode on the document can be 
scanned to play the associated media content segment on a display device. (See Klemmer page 
89 Abstract and page 92 in the 1 st paragraph under "Hardware" discussing barcode scanning 
and video playback.) 

Regarding claim 12: Klemmer teaches wherein the graphical representation includes 
audio content displayed as an audio waveform timeline. (See Klemmer page 92 Fig. 3 showing 
barcodes associated with Voices augmented paper transcripts in the context of the bottom 
paragraph in the right column discussing the adding of barcodes to time code metadata 
information.) 
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Regarding claim 13: Klemmer teaches wherein the timeline includes markers along its 
length that correspond to user-selected segments of media content. (See Klemmer page 92 Fig. 
3 showing barcode markers associated with Voices augmented paper transcripts in the context of 
the bottom paragraph in the right column discussing the adding of barcodes to time code 
metadata information.) 

Regarding claim 14: Klemmer teaches wherein the timeline includes markers along its 
length that correspond to segments of audio content, the segments being defined by a search 
for particular features within the media content. (See Klemmer page 92 Fig. 3 showing 
barcodes associated with Voices augmented paper transcripts in the context of the bottom 
paragraph in the right column discussing the adding of barcode markers to time code metadata 
information.) 



Regarding claim 15: Klemmer teaches wherein the timeline includes markers along its 
length that correspond to segments of media content, at least one of the markers having text 
information describing the segment of media content. (See Klemmer page 92 Fig. 3 showing 
barcodes and associated textual passages.) 
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Regarding claim 16: Klemmer teaches wherein the timeline includes markers along its 
length that each correspond to a segment of the media content, at least one of the markers 
having timestamp information describing the segment of the media content. (See Klemmer 
page 92 Fig. 3 showing barcodes associated with Voices augmented paper transcripts in the 
context of the bottom paragraph in the right column discussing the adding of barcodes to time 
code metadata information.) 

Regarding claim 17: Klemmer teaches wherein the media representation includes a 
header describing the media content. (See Klemmer page 91 bottom paragraph in the right 
column discussing the placement of metadata in a header.) 

Regarding claim 18: Klemmer teaches wherein the feature extraction module is 
further adapted to generate the media representation is generated according to format 
specifications included in a data structure. (See Klemmer page 92 top paragraph in the right 
column discussing the creation of a paper layout based upon a transcript.) 

Regarding claim 19: Klemmer teaches wherein the format specifications included in 
the data structure comprise a number of user-definable fields specifying a format of a 
graphical representation of the media content. (See Klemmer page 92 top paragraph in the 
right column discussing the creation of a paper layout based upon a transcript.) 
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Regarding claim 20: Klemmer teaches wherein the format specifications included in 
the data structure comprise a number of user-definable fields specifying a layout of the media 
representation. (See Klemmer page 92 top paragraph in the right column discussing the creation 
of a paper layout based upon a transcript.) 

Regarding claim 21: Klemmer teaches wherein the format specifications included in 
the data structure comprise a number of user-definable fields specifying the media content 
markers included in the media representation. (See Klemmer page 92 top paragraph in the 
right column discussing the creation of a paper layout based upon a transcript.) 

Regarding claim 22: Klemmer teaches wherein the format specifications included in 
the data structure comprise a number of user-definable fields specifying the feature extraction 
techniques applied to the media content. (See Klemmer page 92 in the top paragraph in the 
right column discussing the creation of MPEG-2 and the use of a layout transcript.) 

Regarding independent claim 25: Klemmer teaches A method for generating a 
representation of time-based the method comprising: extracting features from media content; 
(See Klemmer page 92 in the top paragraph of the right column discussing the creating of an 
MPEG-2 video from a video source.) generating a media representation representing the 
features extracted; (See Klemmer page 92 in the top paragraph of the right column discussing 
the making of corresponding JPEG thumbnails.) formatting the media representation according 
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to a representation specification, the formatting including applying the features extracted to 
the media representation extraction information; (See Klemmer page 92 in the top paragraph of 
the right column discussing the creating of a paper layout from a time stamped transcript.) and 
wherein the formatted media representation includes a graphical representation of a timeline 
and a plurality of user selectable identifiers representing the features extracted from the 
media content for selection by a user to play media content segments of a defined length 
associated with each of the features, wherein the plurality of selectable identifiers are linked to 
locations on the timeline. (See Klemmer page 91 in the last paragraph of the right column 
discussing the adding of time code information to the print format. See also page 92 Fig. 3 
showing barcodes linking chronological sections of a book to A/V data and page 92 in the top 
paragraph of the right column discussing the creating of a paper layout from a time stamped 
transcript.) 

However, Klemmer does not explicitly teach the remaining limitations as claimed. 
Graham, though, discloses printing a document displaying the formatted media representation, 
(See Graham Fig. 4A #408 and col. 5 lines 32-50 teaching the printing of a paper reader's 
assistant document having an imprinted thumbnail and teaching additional information as being a 
"discussion" of user interests.) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Graham for the benefit of Klemmer, because to do so allowed a 
designer to implement a mechanism to personalize a document for a particular user, as taught by 
Graham in the Abstract. These references were all applicable to the same field of endeavor, i.e., 
annotation of paper documents with electronic information. 
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Regarding claim 27: Klemmer does not explicitly teach the remaining limitations as 
claimed. Graham, though, discloses wherein extracting features of media content further 
comprises performing keyword searching on the media content. (See Graham col. 7 lines 50-65 
discussing the use of keywords and keyphrases.) 

Regarding claim 31: Klemmer teaches wherein the graphical representation includes 
audio content displayed as an audio waveform timeline. (See Klemmer page 92 Fig. 3 showing 
text data displayed, the particular data chosen to display having been an obvious variant. See 
also page 92 Fig. 3 showing barcodes associated with Voices augmented paper transcripts in the 
context of the bottom paragraph in the right column discussing the adding of barcodes to time 
code metadata information.) 

Regarding claim 33: Klemmer teaches wherein at least one of the user-selectable 
identifiers comprises a barcode printed on the document displaying media representation. 

(See Klemmer page 92 Fig. 3 showing barcodes printed on a video paper document.) 

Regarding claim 34: Klemmer teaches wherein the user scans the barcode to play the 
associated media content on a display device. (See Klemmer page 92 in the 1 st paragraph in the 
right column under "Hardware" discussing the use of a barcode scanner.) 
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Regarding claim 35: Klemmer teaches further comprising generating markers along 
the timeline, the markers corresponding to user-selected media content. (See Klemmer page 
92 Fig. 3 showing barcodes, in the context of page 91 in the bottom paragraph in the right 
column discussing the adding of timecode metadata.) 

Regarding claim 36: Klemmer teaches further comprising generating markers along 
the timeline, at least one of the markers corresponding to features extracted from the media 
content. (See Klemmer page 92 Fig. 3 showing barcodes associated with Voices augmented 
paper transcripts in the context of the bottom paragraph in the right column discussing the adding 
of barcode markers to time code metadata information.) 

Regarding claim 37: Klemmer teaches further comprising generating markers along 
the timeline, at least one of the markers including text information describing the media 
content. (See Klemmer page 92 Fig. 3 showing barcodes and associated textual passages.) 

Regarding claim 38: Klemmer teaches further comprising generating markers along 
the timeline, at least one of the markers including timestamp information describing the media 
content. (See Klemmer page 92 Fig. 3 showing barcodes associated with Voices augmented 
paper transcripts in the context of the bottom paragraph in the right column discussing the adding 
of barcodes to time code metadata information.) 



Application/Control Number: 10/814,844 Page 15 

Art Unit: 2162 

Regarding claim 39: Klemmer teaches wherein printing the media representation 
further comprises printing a header describing the media content. (See Klemmer page 91 
bottom paragraph in the right column discussing the placement of metadata in a header.) 

Regarding claim 40: Klemmer teaches wherein printing the media representation 
further comprises generating a representation in digital format. (See Klemmer page 92 in the 
top paragraph of the right column discussing the creating of an MPEG-2 video from a video 
source.) 

Regarding claim 41: Klemmer teaches wherein printing the media representation 
further comprises printing a representation in paper format. (See Klemmer page 92 Fig. 3 
showing "Books with Voices augmented paper transcripts".) 

Regarding claim 42: Klemmer teaches wherein formatting the media representation 
according to the representation specification further comprises defining a format of the media 
representation using a data structure with format specifications. (See Klemmer page 92 top 
paragraph in the right column discussing the creation of a paper layout based upon a transcript.) 

Regarding claim 44: Klemmer teaches further comprising applying a barcode 
generation algorithm to render a barcode image including identifier information. (See 
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Klemmer page 92 Fig. 3 showing barcodes that link to further information, it being implied that 
such information requires an identifier for the purposes of storing/locating that information.) 

Regarding claim 45: Klemmer teaches further comprising applying a barcode 
algorithm to render a barcode image including timestamp information. (See Klemmer page 92 
top paragraph in the right column discussing the creation of a paper layout based upon a 
transcript.) 

5. Claims 28-29 are rejected under 35 U.S.C. 103(a) as being unpatentable over Scott R. 
Klemmer et al. ("Books With Voices: Paper Transcripts as a Tangible Interface to Oral 
Histories", CHI 2003. Fort Lauderdale, FL, Apr. 5-10, 2003, pp. 89-96, hereafter referred to as 
"Klemmer") in view of Graham et al. (US Patent No. 6,369,81 1, filed Sep. 9, 1998 and issued 
Apr. 9, 2002, hereafter referred to as "Graham") and Ponceleon et al. (US Patent Application 
Publication No. 2003/0187642, filed Mar. 29, 2002 and published Oct. 2, 2003, hereafter 
referred to as "Ponceleon"). 

Regarding claim 28: Klemmer does not explicitly teach the remaining limitations as 
claimed. Ponceleon, though, discloses wherein extracting features of media content further 
comprises performing speech recognition on the media content. (See Ponceleon Abstract and 
paragraph [0003] discussing the use of automatic speech recognition for discovering salient 
sections in a speech transcription.) 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Ponceleon for the benefit of Klemmer in view of Graham, because to do 
so enabled a designer to implement a system to automatically discover salient segments in a 
speech transcript, as taught by Ponceleon in the Abstract. These references were all applicable to 
the same field of endeavor, i.e., automation information retrieval. 



Regarding claim 29: Klemmer does not explicitly teach the remaining limitations as 
claimed. Ponceleon, though, discloses wherein extracting features of media content further 
comprises performing event detection on the media content. (See Ponceleon paragraph [0007] 
discussing the detection of events occurring in the news.) 
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Conclusion 

6. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 
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Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Cam Y Truong/ /Robert Stevens/ 

Primary Examiner, Art Unit 2 1 62 Examiner 

Art Unit 2162 



June 27, 2008 



